Papers with average precision
MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models produce non-existing facts when faced with questions outside their parametric knowledge, which undermines their reliability. |
| Approach: | They propose a method that separates the learning of answer prediction and confidence estimation during fine-tuning on instruction data. |
| Outcome: | Experiments on multiple models and different model sizes show that the proposed method outperforms baselines by up to 25% in average precision. |
Combining Information-Weighted Sequence Alignment and Sound Correspondence Models for Improved Cognate Detection (C18-1)
Copied to clipboard
| Challenge: | a new approach to cognate detection is proposed to capture the remaining similarities between cognate word forms after thousands of years of divergence. |
| Approach: | They propose a method which uses information weighting and sound correspondence modeling to improve cognate detection. |
| Outcome: | The proposed approach improves on the measure of form similarity and distance-based cognate clustering. |
Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on active object grounding from an egocentric perspective is focusing on localizing and tracking active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Approach: | They propose to use a narrated egocentric video dataset to localize and track active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Outcome: | The proposed framework leads to 54% improvement in standard metrics on the TREK-150-OPE-Det localization + tracking task, and >7% improvement in all standard metrics. |
On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent datasets heuristically choose examples to ensure label balance . state-of-the-art models trained on QQP and WikiQA have only 2.4% average precision . |
| Approach: | They show that recent datasets heuristically choose examples to ensure label balance . they instead use active learning to retrieve uncertain points from a large pool of unlabeled utterance pairs . |
| Outcome: | The proposed model improves on QQP and WikiQA by using more informative negative examples. |
COMICORDA: Dialogue Act Recognition in Comic Books (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing work on dialogue act recognition from images is limited to speech balloon segmentation and optical character recognition. |
| Approach: | They propose a novel DA recognition approach for comic books using speech balloon segmentation, optical character recognition and DA classification. |
| Outcome: | The proposed method achieves 98% average precision for speech balloon segmentation and exceeds 70% accuracy for the DA recognition task. |
CoopQ: Cooperative Game Inspired Layerwise Mixed Precision Quantization for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models struggle when the average precision drops below four bits, limiting deployment on resourceconstrained devices such as mobiles, edge sensors, or standard GPUs. |
| Approach: | They propose a game-like game-inspired mixed-precision quantization method which translates these Shapley estimates into a binary quadratic optimization formulation, assigning either 2 or 4-bit precision to layers under strict memory constraints. |
| Outcome: | The proposed method reduces Perplexity by 20 – 80 % across average precisions spanning 4 bit down to 2 bit, compared to methods relying on isolated metrics. |
Paraphrasing vs Coreferring: Two Sides of the Same Coin (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical resources such as WordNet capture synonyms and hypernyms, as well as antonyms, which can be used to refer to the same event when the arguments are reversed. |
| Approach: | They used annotations from an event coreference dataset as distant supervision to re-score heuristically-extracted predicate paraphrases. |
| Outcome: | The proposed model improved modestly but consistently in the two tasks. |
Semantic matching for text classification with complex class descriptions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text classification support zero-shot learning but not both . Existing approaches do not support zero or few-shot, and are insufficient for complex classes . |
| Approach: | They propose a method which rapidly adapts from seen classes to new/unseen ones . they use labels and complex class descriptions to perform zero- and few-shot learning . |
| Outcome: | The proposed method beats baselines on complex class descriptions by 22.48% . it also improves zero-shot learning by 4.29% . |
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. |
| Approach: | They exploit the signals embedded in URLs to label web documents at scale with an average precision of 94.5% across different language pairs. |
| Outcome: | The proposed method can label documents at 94.5% across languages with high precision . the proposed method is useful for low-resource languages with limited resources . |
VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model (2024.acl-long)
Copied to clipboard
Tianyu Chen, Lin Li, ZhuLiuchuan ZhuLiuchuan, Zongyang Li, Xueqing Liu, Guangtai Liang, Qianxiang Wang, Tao Xie
| Challenge: | Existing work on affected package identification is limited by large language models . a recent study shows that 84% third-party packages contain security vulnerabilities . |
| Approach: | They propose a method to use LLM to generate the affected package . they propose supervised fine-tuning, retrieval augmented generation and a local search algorithm . |
| Outcome: | The proposed method has an average precision of 0.806 for identifying vulnerable packages in four most popular ecosystems in GitHub Advisory. |
Varying Vector Representations and Integrating Meaning Shifts into a PageRank Model for Automatic Term Extraction (2020.lrec-1)
Copied to clipboard
| Challenge: | a comparative study for automatic term extraction from domain-specific language using a PageRank graph algorithm with different edge-weighting methods. |
| Approach: | They propose to use a PageRank algorithm to extract automatic terms from domain-specific language using different edge-weighting methods. |
| Outcome: | The proposed model is compared with a PageRank model with different edge-weighting methods. |
Annotating Zero Anaphora for Question Answering (L18-1)
Copied to clipboard
| Challenge: | a large dataset of zero pronouns has been constructed to identify adjunct zero anaphoras . a lack of a dataset covering them has limited our ability to annotate them exhaustively . |
| Approach: | They propose to annotate adjuncts marked by -de in Japanese and a second scheme to annnotate them in a more direct manner. |
| Outcome: | The proposed annotation schemes are more accurate than the first one. |
ActiveAED: A Human in the Loop Improves Annotation Error Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Annotation error detection (AED) methods flag erroneous annotations for human reannotation . but many of these methods assume a final curation step in which a human annotator decides whether the annotation is erranosome . |
| Approach: | They propose an annotation error detection method that can detect errors more accurately by repeatedly querying a human for error corrections in its prediction loop. |
| Outcome: | The proposed method improves on eight datasets spanning five different tasks and can improve accuracy by up to six percentage points. |